Soft-to-hard Model Transition in Clustering: a Review

Clustering analysis often employs unsupervised learning techniques originally developed for vector quantization. In this framework, a frequent goal of clustering systems is to minimize the quantization error, which is aaected by many local minima. To avoid connnement of reference vectors to local minima of the quantization error and to avoid formation of dead units, hard c-means clustering algorithms are traditionally adapted by replacing their hard competitive strategy with a soft adaptation rule, where the degree of overlap between receptive elds is proportional to a monotonically decreasing scale (temperature) parameter. By starting at a high temperature, which is carefully lowered to zero, a soft-to-hard competitive clustering model transition is pursued, such that local minima of the quantization error are expected to emerge slowly, thereby preventing the set of reference vectors from being trapped in suboptimal states. A review of the hard c-means, Maximum-Entropy, Fuzzy Learning Vector Quantization (FLVQ), Neural Gas (NG), Self-Organizing Map (SOM) and a mixture of Gaussians method is provided, relationships between these methods are highlighted and a possible criterion for discriminating between diierent soft-to-hard competitive clustering model transitions is suggested.